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1. Introduction 

Let f2 be a bounded pseudoconvex domain in C™. 
is the complex 



The d (or Dolbeault)-complex 



(1) L 2 (Q) 
where L?, 



Li 



(0,1) V J V ' ^(0,2)V"J ' ' ' ' ' ^(0,n)^V 

jD^(fi) denotes the space of (0, g)-forms on £1 with coefficients in L?(Vl). 

The norm (with corresponding inner product) is || aj dzj\\ 2 — YIjJq \ a j\ 2 dV. 
Here, the prime denotes summation over strictly increasing g-tuples J, and dzj = 



dzj 1 A dzj 2 A 



A dzj . The <9-operator acts via 



n ^ 

Er-i' oaj 
2^ -Q^ dz 3 A dz -l- 



(2) d[^a J dzj i _ 

3 = 1 J 

The derivatives are taken in the distribution sense, and the domain of d consists of 
those (0, g)-forms where the right hand side is in L 2 q+1 \(Cl). d is then a densely 
defined closed operator, and so has an adjoint mapping L 2 Q 9+1 ^(f2) into L 2 q \(ii). 

We denote this adjoint by d . The complex Laplacian is the operator □ = dd +d d, 
acting as an (unbounded) operator on L 2 ^ (fi) . 

The basic L 2 -existence theorem for the <9-Neumann problem (the problem of 
inverting □) goes back to Hormander |H65| : for each q, 1 < q < n, □ = dd + 
d d is an unbounded self-adjoint operator on L 2 Q n (fi) which is surjective and 
(consequently) has a bounded inverse. This inverse is the d- Neumann operator N q . 
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More precisely, N q satisfies the estimate 

(3) IliVH < (— ) \\u\\, ueL*(n), 



where D is the diameter of the domain f2. If a is a 9-closed (0, q + l)-form, then 
d N q+ ia provides the solution orthogonal to the kernel of d (i.e., the solution of 
minimal norm) to du — a. The 9-Neumann operator is also closely related to the 
Bergman projection P q , the orthogonal projection from L, Q > onto its subspace 

consisting of enclosed forms, via Kohn's formula P q = I — d N q+ id. We refer 
the reader to ]FK72| , |Kr92| , |BS99| , |CS99[ | for further details and references on the 
L 2 -theory of the 9-Neumann problem. 

The question we consider in this article is that of compactness of N q . This 
question is of interest for various reasons. We indicate some consequences of com- 
pactness in Section 2 below. In addition, from the point of view of finding necessary 
and sufficient conditions in terms of properties of the boundary, compactness ap- 
pears to be more tractable than global regularity. 

The compactness condition can be reformulated in several useful ways. 

Lemma 1. Let be a bounded pseudoconvex domain, 1 < q < n. Then the fol- 
lowing are equivalent: 

1. The d -Neumann operator N q is compact from L 2 to itself. 

2. The embedding of the space Dome? fl Dome) , provided with the graph norm 
u — > \\du\\ + \\d u\\, into L 2 q ^(^) is compact. 

3. For every e > there exists a constant C e > such that 

\\uf < e (pu|| 2 + \\d*uf) + CelMlii 

when u € Dom d n Dom d . 

4. The canonical solution operators d N q : L^ — > i? an d 

d*N q+1 : L 2 g+1) (Q) -> L 2 9) (O) are compact. 

The statement in (3) is called a compactness estimate. In (4), we refer to d N q 
and d N q+ i as "canonical solution operators" , although strictly speaking the solu- 
tion operators are the restrictions to the kernel of d. However, d N q and d N q+ \ 
are zero on the orthogonal complement of ker9 9 and kerdq+i, respectively. In par- 
ticular, compactness is not affected. Note that saying that the canonical solution 
operator is compact is the same as saying that there exists some compact solution 
operator, as the projection onto the orthogonal complement of the kernel of d pre- 
serves compactness. The equivalence of (2) and (3) is in | KN65 1 , Lemma 1.1. That 
(1) is equivalent to (2) and (3) is an easy consequence of the general L 2 -theory and 
the fact that L 2 0( ^(J7) embeds compactly into W^ 1 ^ (f2). Finally, the equivalence 

of (1) and (4) follo ws fro m the formula N q = (d* N q )*(d* N q ) + (d* N q+1 )(d* N q+1 )* 
(see jFK72| , p.55, |Ra84| ). 

It is useful to know that compactness is a local property. Roughly speaking, 
the 9-Neumann operator N q on f2 is compact if and only if every boundary point 
has a neighborhood U such that the corresponding 9-Neumann operator on [/nO 
is compact. For simplicity, we will assume that fl and U are such that these 
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intersections are domains (i.e., connected); this is not a problem in the applications 
we have in mind (see section 5). 

Lemma 2. Let O be a bounded pseudoconvex domain in <C n , 1 < q < n. 

1. If for every boundary point there exists a pseudoconvex domain U such that 
N q on (the domain) U n f2 is compact, then N q on Q is compact. 

2. If U is smoothly bounded and strictly pseudoconvex and U f2 is a domain, 
then if the d-Neumann operator on (0, q) -forms on is compact, so is the 
corresponding d-Neumann operator on U 

The proof of (1) is straightforward and results from a partition of unity argument 
together with the interior elliptic regularity of d £B d . The proof of (2) is also 
standard, but it needs some ideas discussed in Section 3 below. Accordingly, we 
postpone this proof; it will be given at the end of Section 3. 

2. Some consequences of compactness 
In the case of smoothly bounded domains, a classical theorem of Kohn and 



Nirenberg KN65 asserts that compactness of N q (as an operator from L^ q \(Q) 
to itself) implies global regularity in the sense of preservation of Sobolev spaces. 
We denote by W S (Q) the standard L 2 -Sobolev spaces on Q, and by Wf Q the 
spaces of (0, g)-forms on with coefficients in W s (tt). 

Theorem 3. Let f2 be a bounded pseudoconvex domain in C" with smooth bound- 
ary. Let 1 < q < n. If N q is compact on L^ ^(fl), then N q is compact (in 
particular, continuous) as an operator from W? ^ (Q) to itself, for all s > 0. 

Remark 1. In Theorem ||, the implication in the other direction is valid as well: 
if N q is compact as an operator from W^ q ^ (O) to itself for some s > 0, then 
N q is compact in L^ q \[&) (and hence in W? 0q JCl), for all s > 0). This follows 
from an abstract theorem about compact operators over a Banach space which are 
symmetric with respect to a scalar product. Such operators are necessarily compact 



with respect to the norm given by the scalar product (see, e.g., [La54|, Corollary 2) 



The Fredholm theory of Toeplitz operators is an immediate consequence of com- 



pactness of the 9-Neumann problem V72 , HI97]. In fact, compactness of the canon- 



ical solution operators implies that commutators between the Bergman projection 



and multiplication operators are compact |CD97] 



Proposition 4. Let 17 be a bounded pseudoconvex domain in C™. Assume that for 
some q, < q < n — 1, the canonical solution operator d Nq+i is compact. Let 
M be a function that has bounded first order partial derivatives on Q. Then the 
commutators [P q , M] between the Bergman projection P q and the multiplication 
operator by M is compact on L^ 9 )(^)- 

A short proof is as follows. For / e L'^ Q ? )(Q), let fi := P q f, f 2 ■= {I - P q )f- 
Since both projections are continuous, it suffices to see that [P q , M] is compact on 
both ker9 and (kerd)' 1 . We have 

[Pq, M]fl = PqMh ~ MPqh = Mf x - T N q ^M f x - M h 
(4) = -d*N q+1 ~8(Mh) = -d*N q+1 (dMAf 1 ). 
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(Compare [3a91], proof of Lemma 3 for a similar argument.) Since dM is bounded 
in Q (so acts as an L 2 -bounded multiplier) and since d N q+1 is compact, [P q ,M] 
is compact on ker d. On the other hand, 



[P q , M]f 2 = P q Mf 2 - MP q f 2 = P q Mf 2 



(5) 



P q Md N q . 



Here we have used the fact that P q d MN q 



l df 2 = P q [M, 8 ]{N q+l d)f 2 . 

\df 2 = 0, since the range of d is 



orthogonal to 9-closed forms (note that N q +± maps into the domain of d , which is 
preserved by multiplication with M). Since [M, d ] acts as a zero order (bounded) 
operator on N q+1 df 2 , and N q+1 d is compact (since N q+ \d — (d N q+ i)*), we are 
done. 

Remark 2. In light of Lemma 1, (4), and Proposition |], compactness of either 
N q or N q+ i implies the compactness of [P q ,M]. It would be of interest to see to 
what extent properties of commutators between the Bergman projections and mul- 
tiplication operators can be used to actually characterize compactness properties 
of the 0-Neumann problem. Compactness of the commutators [P q , M) does imply 
compactness of the canonical solution operator d N q+ i restricted to forms with 
holomorphic coefficients. This is essentially Lemma 2 in [3a91]. Indeed, suppose 
that u G L? 0q+1 JQ) has holomorphic coefficients. Write u — J2jJ2'i< Ujxdzj A 
dz K , 1 < j < n, \K\ = q, then by (§j) above, -Y,j[ p q> ZjMYIk u jK dz K ) = 
^2jd N q+ i(^2' K UjKdzj A dzx) = d N q+ \u. Therefore, d N q +% is compact on 
forms with holomorphic coefficients if the commutators [P q ,~Zj], 1 < j < n, are. On 
convex domains, compactness of this restriction is sufficient to give compactness of 
JVq+i, see ]FS98[ , Remark (2), Section 5. 



Whether or not the <9-Neumann problem is compact has further ramifications 
for the theo ry of the Toeplitz C*-algebras naturally associated to a domain, see 
Sa9lj |Sa95| . 



3. Sufficient conditions for compactness 

We s tart wit h an i nequality due to Catlin that is central to the subject (see 
Ca84a|, p. 45; |Ca87|, Theorem 2.1; see also [BS99 , Section 2, for a somewhat 
different approach to this type of estimate). Assume for the moment that the 
boundary of f2 is sufficiently smooth, say C 2 , to allow integration by parts. Let 
A G C 2 (H) be normalized so that < A < 1. Then 



(6) 



E'E 



K 



d 2 x 

Q dzjdzk 



UjKUkKdV < e ( \\du\f + \\d 



•I") 



for all u G Dom d D Dom d C L^ Q 



(Q). Heuristically, in view of (3) in Lemma [I] 
above (the compactness estimate) , (g) shows that functions A with "large" Hessians 
are desirable when compactness is the goal. To make this precise, it is convenient 
to recall the following notions (see lCa87| ). 

Let Ai°' 9) be the space of (0, g)-forms at z equipped with the standard Hermitian 



metric | J2'j u jdzj\ 2 = XX 



j\uj\ 



For a C 2 -smooth function \{z) in a neighborhood 
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of z, let 

H q (X)(z,u)= EgJ 1 "^' u€A<?'*>. 

\K\=q-lj,k=l J k 

Note that Hi (A) is the usual complex Hessian. 

Lemma 5. Let M > be a constant. The following statements are equivalent. 

1. H q {X){z,u) > M\u\ 2 for all u G K ( z ,q) . 

2. For any orthonormal subset {P , 1 < j < <?} of C n , 

q 

j=i 

3. XTie sum of any q eigenvalues of the Hermitian matrix (d 2 X/dzjdzk) is greater 
than or equal to M . 

The equivalence of (1) and (2) is in JCa87| , pp. 189-190. The equivalence of (1) 



and (3) is implicit in [EKE], pp. 137. This lemma is proved by first establishing it 



in the case when the Hermitian matrix (d X/dzjdzu) is diagonal, and then noting 
that statements (l)-(3) are invariant under a unitary transformation. 

In light of Lemma ^, it is useful to have the following definition: For a bounded 
pseudoconvex domain O, we say that M7 satisfies property (P q ) (or just (Pq)) if for 
every positive number M, there exists a neighborhood U of bfl and a C 2 -smooth 
function Aonf7nfi,0<A<l, such that for all z in U D fi, the sum of any q 
(equivalently: the smallest q) eigenvalues of the Hermitian form (dX/dzjdzk)" k=1 



is at least M. This definition is from [Ca84a] (for q = 1). Note that (Pi) => (P 2 ) 



■ ■ ■ => (P n )- With this definition, we can make our heuristic statement from above 
precise. 

Theorem 6. Let O be a bounded pseudoconvex domain in C™, 1 < q < n. Lf the 
boundary offl satisfies property (Pq), then the d-Neumann operator N q is compact. 

For su fficien tly smooth domains, Theorem |g is in proof of Theorem 1 



(se e also |Ca87 ). The boundary regularity requirement was considerably weakened 



in [ HI97 1 , Theorem 1. The second author showed in [3t97], Corollary 3, that no 
boundary regularity at all is needed. 

Remark 3. The differentiability requirement on A can be relaxed. For example, 
assume only that A is continuous with its complex Hessian, interpreted as a current, 
bo unded below by Mdd c \z\ 2 . Then N q is compact, 1 < q < n. This is also contained 
in |t9|. 

The proof of Theorem ^ results from the following considerations. Assume first 
that 6f2 is smooth, so that we can apply (6). We note that by Lemma 0, the 
condition on A (the function in the definition of property (Pq)) implies that 

(7) M\u(z)\ 2 < H q (X)(z,u) 

for all (0,g)-forms u and z G U PI f2. Choosing M rs 1/e, (0) combines with (||) 
to produce the desired compactness estimate, modulo terms that are compactly 
supported ((Q) holds only near the boundary). These latter terms, however, are 
easily handled by the interior elliptic regularity of d ® d (see [ Ca84a[ ). 
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When no boundary regularity is assumed, one would like to obtain (J^) by first 
integrating over approximating subdomains, and then passing to the limit. The 
reason that this is not immediate is that when forms are restricted to subdomains, 
they are in general no longer in the domain of d on the subdomain. This difficulty 



can be overcome by a regularization procedure introduced in |St97| that exploits 
the (9-Neumann operators of the subdomains. We refer the reader to |St97|, proof 
of Corollary 3, for details. 

The simplest examples of domains satisfying (Pi) (hence (P q ), 1 < q) are strictly 
pseudoconvex domains: it suffices to consider a strictly plurisubharmonic defining 
function. More generally, domains of finite type satisfy (Pi). This is far from 
obvious, however. It is a consequence of the analysis of finite type points in [ Ca84b| 



and [ Da82Q , see Ca84a| . But property (P x ) is considerably more general. It is not 



hard to see that if a domain is strictly pseudoconvex except at finitely many points, 
then it satisfies (Pi). In fact, if the infinite type points of the boundary have 2 
dimensional Hausdorff measure zero, then the domain satisfies (Pi) ([ Bo88fl ; [ |5i87 |, 



Remark on p. 310). On the other hand, Sibony found examples of domains with 
(Pi) where the set of boundary points of infinite type has positive measure ([ |5i87 |. 
p. 310). 

Sibony undertook a systematic study of property (Pi ) in the context of general 
compact sets in C™. One of the main tools is Choquet theory applied to the cone 
of plurisubharmonic functions. One can carry out an analogous study of property 
(P q ) by considering a co ne of functions that reflects the condition on the Hessian 
used in Lemma |], as in [ FS98 1 . For an open set U C C", denote by P q {U) the set 



of continuous functions A on U such that for any z £ U and orthonormal set of 
vectors {ii, • • • , t q } in C™, the function 

C = (Cl, • • • , Cq) G C Q ^ \{Z + Cltl + . • • + (qt q ) 

is subharmonic on {£ e C q ; z + C\ti + . . . + ( q t q € U}. That is, P q {U) consists of 
the continuous functions on U that are subharmonic on each g-dimensional complex 
affine subspace. In particular, P\(U) is the set of all continuous plurisubharmonic 
functions and P n (U) is the set of all continuous subharmonic functions. P q (U) 
is a convex cone in C(U) that is closed under taking the pointwise maximum of 
finitely many of its elements. Each function in P q (U) is a locally uniform limit of 
C°°-smooth elements in P q of slightly smaller open sets: this follows from the usual 



mollifier argument. It is easy to check that — 5Z?=i \ z j\ 2 + (l ~ -OS 



Zi 



For a compact subset X of C™, let P q (X) be the uniform closure in C(X) of 
functions that are in P q (U) for some neighborhood U of X. A probability measure 
/i on X is said to be a P 9 -measure for z G X if 



X(z)< [ Xdfi, for all XeP q (X). 
Jx 



Let J q (X) be the Choquet boundary of P q (X), i.e., the subset of X consisting 
of points z € X such that the point mass is the only P g -measure for z. We refer 



the reader to [G78], Chapter 1 for the elements of Choquet theory. In particular, 



characterizations of the Choquet boundary are given in Theorem 1.13 in [G78|. We 
are interested in the situation where the Choquet boundary is all of X. 
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Proposition 7. Let X be a compact subset ofC n . Then the following are equiva- 
lent: 

1. J q (X)=X. 

2. C(X)=P q (X). 



3. -For any z £ X, there exists an r > smc/i i/iai if X r — X f] B{z,r) then 
C(X r )=P q (X r ). 

4. For any M > 0, there exists a function A smooth in a neighborhood of X such 
that < X(z) < 1 and H q {\){z,u) > M\u\ 2 for z e X and u £ xf' q) . 

Proposition is from [|5i87| (Proposition 1.3 and 1.4, compare also [ 5i89| ) when 



1, but the arguments cover the case o f gen eral q as well (in fact, many of the 
arguments hold in the general context of |G7l , Chapter 1). When q = 1, Sibony 



calls a compact set that has the (equivalent) properties in Proposition Q fi-regular. 
Accordingly, we shall use the term B q -regular when q > 1. The significance of 
Proposition in our context stems from (4): when X = bQ, where ft is a bounded 
domain, then (4) is essentially property (P q ) for 60. Actually, property (P q ) requires 
the function A to exist only in a neighborhood of bil intersected with il, rather 
than in a full neighborhood of Ml. However, it is easy to see that on domains with 
relatively minimally regular boundary (for instance, when the boundary is locally 
a graph), the two notions coincide. 

Remark 4. For boundaries of bounded domains which admit a bounded plurisub- 
harmonic exhaustion function (hyperconvex domains), having the (equivalent) prop- 
erties in Proposition |t] can also be characterized in terms of various properties re- 
lated to the potential theory of the cone P q (fl), for example the existence of peak 
functions in P,(fi) n C(Q). For this, see |Si87| , Theorem 2.1, and |Si89[ , Theorem 



2.3, where the case q = 1 is treated. However, Sibony's arguments carry over to 
the case of general q. 

The easiest way that fi-regularity can fail is for the boundary to contain an ana- 
lytic disc: in view of the maximum principle, C(btt) = P q (bfl) cannot hold. A more 
direct argument is to pull back to the unit disc and to observe that subharmonic 
functions on the unit disc, with values between and 1, cannot have arbitrarily large 
Hessians. Similarly, g-dimensional varieties in the boundary are incompatible with 
fig-regularity. However, the absence of varieties is not sufficient for fig-regularity 
in general: Sibony has given examples of complete smooth Hartogs domains in C 2 
whose boundaries contain no discs, yet are not _B-rcgular. ([ [Si87 |, p. 310). It turns 



out that on locally convexifiable domains, fig-regularity (hence property (P q )) is 
equivalent to the absence of g-dimensional varieties from the boundary; we will 
discuss this in section 5 below. 

fig-regularity is preserved under countable unions. 

Proposition 8. Let X^, k = 1,2, 3, . . ., be compact subsets o/C", all of which are 
B q -regular. If X = Uj^Afc is compact, then X is B q -regular. 

For q — 1, this is Proposition 1.9 in [|5i87 |. Sibony's proof works essentially 



verbatim to give the result for all q. (This includes the supporting Propositions 1.4 
and 1.6, and Lemma 1.8.) 

We conclude this section by completeing the proof of Lemma from Section 1. 
Assume that there is a compactness estimate for (0, g)-forms on £1, and let U be 



8 



SIQI FU AND EMIL J. STRAUBE 



strictly pseudoconvex (and U(lQ a domain). We show how to obtain a compactness 
estimate on U n Q. Let u G Dom3 n DomQ* C LiL JU n 0). Fix e > 0. The 
boundary of U is strictly pseudoconvex, so satisfies property (Pi), hence property 
(P q ) for all q. Thus for a smooth cutoff function <j> t that is identically equal to 
1 on bU and supported in a sufficiently small neighborhood of bU, we get by the 
arguments in the proof of Theorem [(]: 



IMItrnn ~ ll^llunn + 11(1 ~ <Pe)u\\una 

< e (p(0 e «)||^nn + P*(<Mllann) + 11(1 - 4>>f um 

£ e (P«llann + P*«llifno) + CA\xA\unn + 11(1 - ^Mlunn- 

Here, % e is a smooth cutoff function identically 1 on the support of Sj4>e whose 
support is compact in U, and C e := max | y <f> e \. Now view the forms XeU and (1 — 
4> t )u as forms on O; they are in the domain of d on fl. Applying the compactness 
estimate on f2, with an e' sufficiently small, yields the desired compactness estimate 
on U D fl (terms involving ||u|| 2 can be absorbed if e' is chosen small enough). This 
completes the proof of Lemma |[ 

4. Discs in the boundary vs. compactness 

In this section, we restrict our attention to compactness of the 9-Neumann 
operator on (0, l)-forms. Since a disc in the boundary is a blatant violation of 
-B-regularity, and given that discs are known to prevent hypoellipticity ( [|Ca81 |, 



|DP81p , it is natural to ask whether there is a corresponding failure of compact 



Here is a simple example (taken from [Li85 , Kr8S|). Let O be the bidisc in 



C 2 , with the edge rounded. Denote by D the unit disc in C, and let / € L 2 (D) D 
ker<9. Using subscripts to denote L 2 -norms on domains, we have that ||^2/(^i)||n ~ 
ll/ll-D ~ ll/lln- Also, d(z2f(zi)) — f{z\)dz2, and since 22/(21) is orthogonal in 
L 2 (Vl) to the holomorphic functions, it is the canonical solution, i.e., 22/(21) = 
d Ni(f(z\)dz2). Since the norms compare, compactness of d N\ (hence of iVi 
by (4) in Lemma 1) would imply compactness of the unit ball in L 2 (D) n keid. 
Consequently, Ni is not compact. 

On the other hand, here is an example of a pseudoconvex Reinhardt domain in 
C 2 with a disc in the boundary, but with compact 9-Neumann operator N±. The 
domain is incomplete and non- smooth. Let Q = {(21,22) G C 2 ; |zi| 2 + |z 2 | 2 < 
1, < \zi\ < 1}, i.e., £1 is the unit ball in C 2 minus the variety {z\ = 0}. The point 
is that the L 2 -theory does not detect the deletion of this variety, and so is the same 
as on the unit ball. More precisely, the natural isometry L 2 q -\($i) <-^> L 2 q \(B) [B 

denotes the unit ball) commutes with d q , q = 0,1,2. This can be checked by an 



argument completely analogous to that in [Be82|, p. 687. But then the d operators 
have similar commutation properties, hence so do the □ operators and their inverses, 
the 5-Neumann operators. In particular, N\ on Q inherits compactness from Ni 
on B. 

When some boundary regularity is assumed, the phenomenon in the above ex- 
ample cannot occur on domains in C 2 . 
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Proposition 9. Let fl be a bounded pseudoconvex domain in C 2 with Lipschitz 
boundary. If the boundary of Q contains an analytic disc, then the d-Neumann 
operator N\ on Q is not compact. 

As of this writing, it is not known whether Proposition ^ holds in higher di- 
mensions. Proposition |^ has been part of the folklore for many years; it is usually 



attributed to David Catlin; Michael Christ has also found a proof [Ch98| 



A proof of Proposition H results from adapting the ideas in JCa81 | and |DP81| 



We have also used these ideas in [FS9S], Section 4. By Lemma lL (4), it suffices to 
show that the canonical solution operator d Ni is not compact. We first assume 
that the disc in the boundary is an affine disc, say the disc D 2rn x {0}, where 
D r := {z e C; \z\ < r}. After a suitable complex linear change of coordinates, 
fl is defined near the origin by y 2 < p{zi,X2) (z,j — xj + iyj, j = 1,2), where p 
is a Lipschitz function. Since p(zi,0) = for z\ <E D2r , \p(zi,x 2 )\ — \p(z\,x?i) — 
p{z\,G)\ < C\x2\ when \x%\ is sufficiently small. Therefore, there exist circular 
wedges Wo and W\ of the same radius in the Z2-plane, symmetric about the y 2 - 
axis, such that D ro x Wo C O, and £1 n (D ro x D ri ) C D ro x W\, for a suitable 
r\ > (shrink ro if necessary). We first observe that for any r^ > smaller than 
the (common) radius of Wo and W\ , there is a sequence of holomorphic functions 
in Wx, bounded in L 2 (Wi), so that no subsequence converges in L 2 (WoC] D r3 ); i.e., 
the restriction operator from the Bergman space on W\ to the Bergman space on 
Wq H D r3 is not compact. This is easily seen by considering the sequence 



fj (z 2 ) : , ^-±±,r V 

V a 

(defined via a branch cut along the positive imaginary axis), where aj /* 1. Here, R 
is the radius of the wedge Wt, and a is its angle. Denote by Xi(t) a smooth cut-off 
function that is identically 1 for < t < Tq/2, and identically for t > 3ro/4, and by 
X2(t) a cut-off function that has these properties, but with 7*1 instead of ro- Then 
the forms aj := d(fj(z2)xi(\ z i\)X2(\ z 2\)) are 9-closed on fi. Let gj := d N\a.j. 
After passing to a subsequence, we may assume that {gj}JL\ converges in L 2 (f2) 
if d Ni is compact. Set hj(zi,z 2 ) := Xi(\ z i\)X2(\z2\)f](z2) - g 3 (zi,z 2 ). Then the 
hj are holomorphic in Q, and hj = —gj outside the support of Xi(l z i|)X2(|-Z2|) 
(and hence {hj}°°^ 1 converges in L 2 of fl minus that support). Using the mean 
value property of holomorphic functions in the z\ variable gives that \hj\°yL\ also 
converges in L 2 {D ro / 2 x (Wo H D rs )). This convergence also holds for {gj}j^ =1 , 
hence for {xi (l-^i |)X2 (1-^2 (-Z2)}^i ■ But this implies that {fj}j^i converges in 
Wo H D rs for any r% < r%/2, a contradiction. Therefore, d Ni is not compact. 

The general case of Proposition ^ is obtained by replacing the products disc x 
wedge above by suitable biholomorphic images of such products: the crucial non- 
compactness of the restriction operator between the Bergman spaces is invariant 
under these biholomorphisms. 

Whether there can be obstructions to compactness (say for (0, l)-forms) more 
subtle than discs in the boundary has been settled only recently. Peter Math- 



eos [ M97 1 showed that there exist smoothly bounded complete Hartogs domains in 
C 2 without discs in their boundaries, but whose 9-Neumann operators are nonethe- 
less not compact. Such a domain fl will be of the form fl — {(z, us) £ C 2 ; z € 
Hi, M < e-*( z )}, where fii is a domain in C and $ is smooth and subharmonic in 
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f2i. Assume that the boundary points of the form (z, 0) are strictly pseudoconvex. 
The weakly pseudoconvex points are the points {(z,w) <= bfl; A<E>(z) = 0}. It is 
easy to see that bfl will contain an analytic disc if and only if the set W := {z S 
fii; A$(z) = 0} has non-empty interior (to produce an analytic disc, consider 
a conjugate harmonic function to $ on a (small) disc contained in the interior of 
W). The boundary will fail to satisfy property (Pi) (equivalently: P-regularity) if 
and only if W has non-empty fine interior, see J5i87 ], p. 310. Recall that the fine 



topology is the smallest topology that makes all subharmonic functions continu- 



ous; see, e.g., [Hc69 for properties of this topology. It is strictly larger than the 



Euclidean topology, and there exist compact sets with empty Euclidean interior, 



but non-empty fine interior. Matheos [M97| constructed examples of compact sets 



W with empty Euclidean interior and associated complete Hartogs domains whose 
weakly pseudoconvex boundary points project onto W and whose <9-Neumann oper- 
ator Ni is not compact. (Such sets W then necessarily have nonempty fine interior, 
by Sibony's result). Matheos' ideas combine with results from potential theory to 
give the following: Every compact set W C C with empty Euclidean interior but 
non-empty fine interior arises in this way. 

Theorem 10. Assume that W is a compact subset o/C with non-empty fine inte- 
rior. Then there exists a smooth, bounded, pseudoconvex, complete Hartogs domain 
Q in C 2 whose weakly pseudoconvex boundary points project onto W and whose 
d-Neumann operator N\ is not compact. 

Remark 5. In particular, when W has empty Euclidean interior, the complete 
Hartogs domain will have no discs in the boundary. Note that it does not follow 
from Theorem [l^ that if the boundary of a smooth Hartogs domain does not satisfy 
property (Pi) (equivalently: the associated set W in the plane has non-empty fine 
interior), then its 9-Neumann operator N\ is not compact. Whether or not this 
implication is true (that is, whether or not for complete smooth Hartogs domains 
in C 2 , property (Pi) and compactness of N% are actually equivalent) remains open. 

Remark 6. Although the Hartogs domains in Theorem [n] have non-compact 



9-Neumann operator, this operator is regular in Sobolev spaces; see BS92], Theo- 
rem 1: any bounded smooth Hartogs domain in C 2 with no disc in the boundary 



is "nowhere wormlike", in the terminology of BS92|, and so has globally regular 
9-Neumann operator N±. 



Matheos first uses the machinery of Chang, Nagel, and Stein [CNS92| to show 
that a compactness estimate on is equivalent to a compactness estimate on the 
boundary, in terms of db and d b . This equivalence is established for all smooth 
pseudoconvex domains in C 2 . Then, the key property of the set W is that there 
exists a sequence of "Rayleigh functions" with supports shrinking to W. This last 



property turns out to hold for every compact set with non-empty fine interior (|F99|, 
section 3). We now indicate how to combine this fact with Matheos' arguments to 
prove Theorem [l0[ We work on fl directly; this results in some simplifications. 

For an open set ft in E", let A(fi) = inf{||V/|| 2 ; / € W^(fl), ||/|| = 1}. The 
classical Rayleigh- Ritz formula says that A(f2) is the first eigenvalue of the Dirichlet 
problem for the (negative) Laplacian if £1 is regular with respect to the Dirichlet 



problem (see, e.g., [Cha84|). The Dirichlet problem has also been formulated for 



finely open sets in R™ (cf. |F72|). Among many properties, the monotonicity of A 



with respect to the domain remains true for finely open sets. That is, if Q2 D ^1 7^ 



COMPACTNESS IN THE d-NEUMANN PROBLEM 



11 



are finely open sets in R ra , then A(fi2) < A(fii) < oo. We refer the reader to |F99|, 
section 3, for a discussion of these (and further) results concerning the Dirichlet 
Laplacian on finely open sets. 

Without loss of generality, we assume that W is contained in the open unit disc 
D. Let {Dj}f =1 be the connected c ompon ents of D\W. Let W k = D \ (u£ =1 I3 )) . 
It follows from Theorem 10.14 in |He69[| that Wk D int/(W) ^ 0, where int/ 
denotes the interior in the fine topology. Therefore X(Wk) < A(int/(W)) < oo. It 
follows that there exists a (Rayleigh) sequence of functions Vk & C^{Wk) such that 
||ufc|| = 1 and ||VUfc|| < 1. (Throughout the rest of this section, A(k) < B(k) means 
that A(k) < C ■ B(k), where C is some positive constant independent of k.) 

Let ip be a non-negative smooth function on C that vanishes to infinite order on 
W and is strictly positive on C\W. Let ipj = ip on Dj and ipj = on D\Dj. Then 
p = X^jli V'j on D an d V'j € C°°(D). We now choose by induction a sequence 
{ c j}j2=i in (0, 1] and a strictly increasing sequence of positive integers {n,j}°^ 1 
such that the functions ipj = Cjipj satisfy the following: (1) rij J D ijjjdA = 2tt; 

(2) < ||^||l~; (3) nj\\i) J+1 \\ L ^ < 1; (4) n j+1 is divisible by rij. First 

choose ni sufficiently large so that C\ = 2^j(nx J D ip\dA) < 1. Suppose that we 
have chosen Cj and rij. We can choose an integer m > 2 sufficiently large so that 
Cj+i =2Tr/(m-njJ D . +i i/j j+ idA) < min{l, ||^|| L =o/||^ + i||z~, V(«jIIV'j+iIU°=')}- 
We then let Jij+i = m ■ rij. 

We now let p = YljLi Then ip G C°°(D). (The supports of ^ = are 
disjoint, except for points in W, where all derivatives vanish. Consequently the 
series actually converges in, for example, Sobolev norms.) Let G = (l/2rr) log \z\ 
and let $(z) = J D G(z - w)lp{w)dA{w). Then $(z) e C°°(S) and A$ = <p. Note 
that <I> is strictly subharmonic near dD. Extend <I> to the disc D with center and 
radius 2 such that $ is smooth on D, strictly subharmonic on D \ D, and equals 
— (l/2)log(4 — \z\ 2 ) near dD. (Extend $ smoothly to a function $i compactly 
supported in D. Then add a radial function that is identically for \z\ < 1 and 
agrees with —(1/2) log(4— \z\ 2 ) when \z\ is close to 2. Such a function can be chosen 
to have its second (radial) derivative as big as we wish on a given compact subset 
of {1 < \z\ < 2}, in particular on {A$i < 0}.) Then fi = {(z,w); z € D, \w\ < 
exp(— $(z))} is smooth. It remains to see that the 3-Neumann operator of f2 is not 
compact. For this, we shall construct a sequence of (0, l)-forms that contradicts a 
compactness estimate, i.e., we construct a sequence {uk}^! in Dom(<9*) n C°°(fi) 
such that the estimate 



(8) IK-H 2 < e(\\du k \\ 2 + \\d*u k f) + CelKH 2 -!, 

uniformly in k, will fail for some e > 0. 

Let <p k = J2j=i "4>k and let $ fc = J D G(z - w)<p k (w)dA(w). Then A$ fe = tp h = 

on Wfe. It follows from the way we construct ipj that n^k has a harmonic conjugate 
Ofe on VFfc whose values are determined up to an integer multiple of 2ir. There- 
fore exp(z0fc) is single valued and smooth on Wk- Let /& = ■JrTkVk(z) exp(nfc4> + 
iefe)?/;" 1 - 1 andletu fe = / fc -(wdz-2exp(-2$)$ z dw)). Then u k £ Dom(d*)nC°°(n) 
and ||m/c|| 2 ~ ||/fe|| 2 ~ L Moreover, {/&} and hence {itfc} converge weakly to zero 
in L 2 (f2), and L 2 ^(fi), respectively (note that fj _L fk in L 2 (f2) when j 7^ k) and 
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so {u k } converges to zero in W^-^iil) in norm. (Alternatively, one easily checks 
that || || ?_x ~ ||/fc||-i (l/ n fc)ll/fell 2 ~ l/ n fe ■) Therefore, in order to see that 
estimate (||) is violated for some e > 0, it suffices to see that ||<9ufc|| 2 + ||c?*itfe|| 2 is 
bounded independently of k. The Kohn-Morrey formula gives 



du k \\ 2 + \\d*u k \\ 2 = [ V J-£^( Uk ) j (u k )td*+ [ V 



d{u k )i 



dzj 



2 



dV, 



where (temporarily) z\ = z, Z2 — w. p is a defining function normalized so that 
| V p\ = 1 on M7. We may take this function to be ww — exp(— 2$(z)), suitably nor- 
malized. Then the right hand side is dominated by a positive constant (independent 
of k) times 



/ ® z - z \fk\ 2 da+ f (\f k \ 2 + \h- z \ 2 )dV 
Jan Jn 



< / {n k \v k \ 2 ^zz + \vk\ 2 + \v k z\ 2 + \v k (n k ® + i& k ) \ 2 )dA. 
Jw k 

The integral of the first term in the last expression is not greater than n-fc||^||i<=«=(vi^ fc ) 
x f w \vk\ 2 dA = nk\\ipk+i\\i,° e (w k ) ^ 1- The contribution from the second and third 
terms is under control because of our choice of the sequence {vk}kLi- The integral 
of the last term is not greater than || (n k ^ + *@fc)z|| 2 j oo( H/fc - ) - Since (n k ^ + i&k)z — 
n k ($ — &k)z for z € Wk (because (n^k + «0fe)2 = 0), and the modulus of the 
latter term is not greater than 

■j— [ t- — ^j-dA(w) < n k \\(f- <Pk\\L°°(D) = "fell^fc+illi^fw*) < 1) 
An J D \z-w\ 

it follows that ||9ufc|| 2 + ||5 Mfc|| 2 < 1. This concludes the proof of Theorem |l0|. 
5. Domains related to convex domains 

The sufficient conditions for compactness discussed in Section 3 are also necessary 
in the case of convex domains, and, more generally, in the case of domains that are 
locally convexifiable. Moreover, for these domains, these conditions are equivalent 
to the absence of (germs of) varieties from the boundary. Pseudoconvex Reinhardt 
domains are "almost" in this latter class, that is, they are locally convexifiable at 
the boundary points away from the coordinate hyperplanes (but may not be so at 
boundary points on the coordinate hyperplanes, see, e.g., | FIK96| ), and some of 
the results carry over to this class. 

We say that a domain is locally convexifiable if for every boundary point there 
is a neighborhood, and a biholomorphic map defined on this neighborhood, that 
takes the intersection of the domain with the neighborhood onto a convex domain. 
Note that boundaries of convex domains are locally graphs of Lipschitz functions, so 
the same is true for t he bou ndary of a locally convexifiable domain. The following 
theorem comes from [ FS98f . 



Theorem 11. Let CI be a bounded pseudoconvex domain in C" which is locally 
convexifiable, let 1 < q < n. The following are equivalent: 

1. The d-Neumann operator N q is compact. 

2. The boundary of fi does not contain any analytic variety of dimension > q. 
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3. The boundary of fl satisfies property (P q ). 

Note that for a locally convexifiable domain, property (P q ) is equivalent to ir- 
regularity. Henkin and Iordan HI97 had shown earlier that in the case of convex 
domains, if the boundary contains no analytic varieties of dimension > 1, then all 
<9-Neumann operators N q , 1 < q < n, are compact. 

Actually, that (2) =>- (3) holds for the general case of locally convexifiable do- 
mains is only implicit in [FS98]. We now indicate how to make this explicit. By 
Proposition [?], property (P q ) for bfl will follow if we show that for any z G bfl, the 
point mass is the only P 9 -measure on bfl at z. This last property will follow if we can 
establish it locally again by Proposition [j]. So, let now K be a compact subset of the 
boundary, small enough so that it is contained in some open set on which there is a 
biholomorphism that convexities the boundary, say w = f(z) = (/i(z), . . . , f n (z))- 
By shrinking K if necessary, we may assume that there is a (small) domain U such 
that K C U, f(U fl fl) is convex, / is biholomorphic in a neighborhood of U n ft, 
and there is no variety of dimension greater than or equal to q in the boundary of 
f(U H fl). By [FS98|, Proposition 3.2, there exists, through each boundary point, 
a complex affine subspace L of dimension < q — 1 such that L PI f(U PI fl) is a peak 
set for A{f{U fl fl)), the algebra of functions analytic in f(U fl fl) and continuous 
on f(U fl 0). The inverse image under / of L gives a manifold of dimension < q— 1 
such that n U fl ft is a peak set for A(U fl fi). For every boundary point in 

K, there exists a peak set for A(Unfl) of this form that contains the point. Let g be 
the corresponding peak function. Note that every element in A(U fl VI) can be ap- 
proximated uniformly on U D O by functions analytic in a neighborhood of U D f2, 
since this is true for the corresponding algebra on the convex domain f(U nf2), and 
/ is biholomorphic in a neighborhood of U fl O. Thus |g| € P q (K). Consequently, 
for z £ iiT, all P g -measures of if at z are supported on A'n/ _1 (L). Let now zq £ K, 
and choose coordinates in C n such that /(zo) = and L C {u>g = w q +\ = . . . = 0}. 
Then for sufficiently large M, the function - Y%Zi \fj( z )\ 2 + M YT 1=q \fA z )? is in 
P q (K). It equals zero at zq, and is non-positive on K n / _1 (A). If we integrate 
against a P g -measure for zq over K, and take into account that the measure is 
supported on K n it follows that it is actually a point mass at zq. 



Remark 7. The proof of (1) (2) in FS9S ] establishes the following: if N q is 
compact and Q is convexifiable at (hence near) a boundary point p, then the bound- 
ary cannot contain g-dimensional varieties near p. The reduction to the (globally) 
convex case uses the localization in Lemma |^ in an essential way, together with 
the observation that compactness of the relevant (9-Neumann operators is invariant 
under the convexifying biholomorphism (Lemma 0, (4), and the observation that 
compactness of the canonical solution operator follows once there is some compact 
solution operator), see [ FS98 ], Remark (1) in section 5. 

Remark 8. On convex domains, more is true. As mentioned in Section 2, com- 
pactness of the commutators [P q ^i,Zj] implies compactness of d N q restricted to 
forms with holomorphic coefficients (always) , and this in turn implies compactness 
of d*N q on all of L 2 ^ Jil) if ft is convex ( jFS9§], Remark (2), Section 5). Thus 
in the convex case, properties (l)-(3) in Theorem 11 are furthermore equivalent 
to compactness of the commutators [P q -i,M] discussed in Proposition || as well 
as to compactness of o N q , restricted to forms with holomorphic coefficients. In 
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addition, the absence of varieties from the boundary is equivalent with the absence 
of affine varieties. This is a simple manifestation of the general principle that on 
convex boundaries, questions of orders of contact of varieties are decided by orders 
of contact of affine varieties. (See [ FS9S| ], Section 2, and the references given there.) 



As mentioned earlier, pseudoconvex Reinhardt domains are locally convexifiable 
at most of their boundary points, so it is not surprising that the above ideas also 
give results on this class of domains. 

Theorem 12. Let Q be a bounded pseudoconvex Reinhardt domain in C" , 1 < q < 
n. If the boundary of Q does not contain an analytic variety of dimension > q, then 
it is B q -regular, and consequently, the d-Neumann operator N q on (0, q)-forms is 
compact. 

The fact that if the boundary of a Reinhardt do main contains no varieties of 
dimension > 1, then N q , 1 < q < n, is compact is in [ HI97| , Corollary 1. However, 



there is an inaccuracy in the proof: it is asserted that on a bounded pseudoconvex 
Reinhardt domain Q, every boundary point is a peak point for the algebra A(£l), the 
algebra of continuous functions on Q which are holomorphic on f2. Pseudoconvex 
Reinhardt domains in C 2 lying over the punctured disc show that this is not the 
case in general: the boundary point corresponding to the punctur e is n ot a peak 
point. Likewise, such a domain will not satisfy property (B) from HI97| . 



Remark 9. The example of the domain Q = {(zi,z 2 ) G C 2 ; \zi\ 2 + \z 2 \ 2 < 1,0 < 
\zi\ < 1} discussed in Section 4 above shows that compactness of the 9-Neumann 
operator N q on a Reinhardt domain docs in general not imply the absence of va- 
rieties from the boundary. It does imply, in view of Remark 7, that the bound- 
ary intersected with the complement of the coordinate hyperplanes cannot contain 
q-dimensional varieties if N q is compact. (The implication that if iVi is compact, 
then the boundary (away from the coordinate hyperplanes) cannot contain analytic 
discs is contained in work of Salinas concerning Toeplitz C*-algebras on Bergman 
spaces. For this work, and further references, see the two surveys ]Sa91 , 3a9f;].) 



We now prove Theorem |l2]. The proof is by induction on the dimension n, as 
follows. When n = 1, Q is either a disk, a punctured disk, or an annulus. In all three 
cases, the boundary is £?-regular. We assume now that the theorem holds for all 
dimensions < n — 1, and show that it then holds in dimension n. Since the domain 
17 is pseudoconvex, the boundary is locally convexifiable near each boundary point 
(zi, . . . , z n ) where none of the Zj (1 < j < n) are 0. As in the proof of Theorem [Tl| 
(the implication (2) =>■ (3)), it follows that for every such boundary point z, there 
is r > such that the compact set bfl fl B(z, r) is i? 9 -regular. Consequently, the 
part of the boundary where all coordinates are non-zero (this is not a compact set) 
is a countable union of compact sets, all of which are i? g -regular. 

The rest of the boundary is a finite union of compact sets of the form bVLC\{zj 1 = 
= °) ■%•*,+! 7^ 0,... ,z jn ^ 0}, where {31,32,- ■■ ,3n} is a permuta- 



z 



tion of {1,2,... ,71} and k ranges from 1 to n — 1. (The case k — n results 
in only one set, bft D {0}, which is either empty or {0}. But the set {0} is 
-B g -regular.) So fix (ji,... ,jk)- Assume first that n— k > q. We will show 
that bQ H {zj ± = . . . = Zj k = 0, zj k+1 7^ 0, . . . , Zj n 7^ 0} (which we may as- 
sume nonempty) is the boundary, in {zj 1 = ... = Zj k = 0} = C" _fc , of the 
bounded Reinhardt domain n {zj ± = ... = Zj k = 0}. Since f2 is pseudocon- 
vex, its logarithmic image, log|fi| := {(log \zi\, . . . ,log|z„|); z £ fi, Zj 7^ 0, 1 < 
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j < n}, is convex. Since there is a boundary point with a zero coordinate, it 
is unbounded. We use the tools and notions developed in Section 8 of [Ro70|. 
Let {z v }^Li ^ e a sequence of points in Q that converges to a boundary point 
(0, . . . ,0, Zfc+i, . • • , z„), with Zk+i 0, . . . , z n 0. (Without loss of generality, we 
assume that (ji, j2, . . . ,j n ) — (L 2, . . . , n).) It follows from the convexity of the 
unbounded set log |0| (specifically, see [Ro70 , Theorem 8.2 and Theorem 8.3) that 
there is a vector r — (rj., . . . , r^, 0, . . . , 0) € R n such that (+r also belongs to log |fi| 
for each £ G log |f2| (r is in the recession cone of log |Q| in the terminology of [ Ro70| ). 
This shows that the projection of f2 onto {zi = . . . = z% = 0} consists entirely of 
points in f2. Since the boundary of Q contains no g-dimensional varieties, and since 
n — k > q, at least one of these points has to be an interior point of f2. Pseudo- 
convexity now implies that f2 is complete with respect to the variables (z\, ■ ■ . , Zk). 
Therefore, the projection of fl onto {z\ = . . . = Zk = 0} equals the intersection of 
f2 with this this coordinate "plane". Consequently, Ml n {z\ = ... = Zk = 0} is 
the same as the boundary of £1 n {z\ = . . . = z% = 0}. This is the boundary of a 
pseudoconvex bounded Reinhardt domain in C n ~ k . It contains no g-dimensional 
varieties. By the induction assumption, it is i? g -regular in C n ~ k . By consider- 
ing functions of the form \(zk+i, ■ . ■ , z n ) + A/(|zi| 2 + . . . + |zfe| 2 ), one sees that 
bfl n {21 = . . . = Zk = 0} also is B g -regular as a subset of C". 

It remains to consider the case n — k < q. In this case, any compact subset of 
{zi = . . . = Zk = 0} is S 9 -regular as a subset of C™ (consider functions in P q (C n ) 
of the form M{\ Zl \ 2 + . . . + \z k \ 2 )). 

We now know that bQ, is a countable union of compact, -B q -regular subsets of 
C™. Therefore, by Proposition ||, Ml itself is £? 9 -regular as well. 

Remark 10. Recall the second example from Section 4 above: is the unit ball in 
C 2 minus the variety {z\ = 0}, and Ni is compact on ft. This case is not covered 
by Theorem |lj (the boundary contains a disc). It is instructive to compare the 
two situations. In the above proof of Theorem [l2|, we have used the fact that bfl 
contains no (/-dimensional variety twice. Namely, we used it first to conclude that 
the projection of fl onto {z\ = • • • = Zk = 0} consists of interior points of 0. This 
use of the absence of varieties is merely convenient: one could "fill in" these points as 
in the example. However, in order to make the induction run, we need to know that 
the boundary of this projection is free of q-dimensional varieties, and this is crucial 
for the argument, unless q — n — 1. When q = n — 1, the latter property comes for 
free. From these ideas, there results the following characterization of compactness 
of N n ^i on a bounded pseudoconvex Reinhardt domain f2 in C n : iV n _i is compact 
if and only if bQ \ {z\ ■ ■ ■ z n — 0} contains no (n — 1)- dimensional variety. For 
complete Reinhardt domains, this characterization was obtained in ffSSU8S|l . 



ACKNOWLEDGMENTS 



Part of this work was done while the first author visited Texas A & M University 
in the summer of 1999. He thanks the Department of Mathematics and Harold and 
Heidi Boas for their hospitality. 



16 

[Be82] 
[B088] 
[BS92] 
[BS99] 

[Ca81] 

[Ca83] 
[Ca84a] 

[Ca84b] 

[Ca87] 

[CD97] 

[CNS92] 

[Cha84] 

[CS99] 

[Ch98] 
[Da82] 

[DP81] 

[FK72] 

[FIK96] 

[FS98] 

[F72] 

[F99] 
[G78] 

[He69] 
[HI97] 

[H65] 

[KN65] 

[Kr88] 



SIQI FU AND EMIL J. STRAUBE 



References 

Steven R. Bell, The Bergman kernel function and proper holomorphic mappings, Trans- 
actions of the American Mathematical Society 270 (1982), no. 2, 585-591. 
Harold P. Boas, Small sets of infinite type are benign for the d-Neumann problem, 
Proceedings of the American Mathematical Society 103 (1988), no. 2, 569-578. 
Harold P. Boas and Emil J. Straube, The Bergman projection on Hartogs domains in 
C 2 , Transactions of the American Mathematical Society 331 (1992), no. 2, 529-540. 
, Global regularity of the d-Neumann problem: a survey of the I? -Sobolev the- 
ory, Several Complex Variables (M. Schneider and Y.-T. Siu, eds.), MSRI Publications, 
vol. 37, to appear. 

David Catlin, Necessary conditions for subellipticity and hypoellipticity for the 8- 
Neumann problem on pseudoconvex domains, Recent Developments in Several Complex 
Variables (John E. Fornasss, ed.), Annals of Mathematics Studies, no. 100, Princeton 
University Press, 1981, pp. 93-100. 

, Necessary conditions for subellipticity of the d-Neumann problem, Annals of 

Mathematics (2) 117 (1983), no. 1, 147-171. 

, Global regularity of the d-Neumann problem, Complex Analysis of Several 

Variables (Yum-Tong Siu, ed.), Proceedings of Symposia in Pure Mathematics, no. 41, 
American Mathematical Society, 1984, pp. 39-49. 

, Boundary invariants of pseudoconvex domains, Annals of Mathematics (2) 

120 (1984), no. 3, 529-586. 

, Subelliptic estimates for the d-Neumann problem on pseudoconvex domains, 

Annals of Mathematics (2) 126 (1987), no. 1, 131-191. 

David Catlin and John D'Angclo, Positivity conditions for bihomogeneous polynomials, 
Math. Res. Lett. 4 (1997), 555-567. 

D.-C. Chang, A. Nagel, and E. M. Stein, Estimates for the d-Neumann problem in 
pseudoconvex domains of finite type in C 2 , Acta Mathcmatica 169 (1992), 153-228. 
Isaac Chavel, Eigenvalues in Riemannian geometry, Pure and Applied Mathematics 
vol.115, Academic Press, 1984 

So-Chin Chen and Mei-Chi Shaw, Partial differential equations in several complex 
variables, book to appear. 

Michael Christ, Private communication, 1998. 

John D'Angclo, Real hypersurfaces, orders of contact, and applications, Annals of 
Mathematics (2) 115 (1982), no. 3, 615-637. 

Klas Diederich and Peter Pflug, Necessary conditions for hypoellipticity of the d- 
problem, Recent Developments in Several Complex Variables (John E. Forna:ss, ed.), 
Annals of Mathematics Studies, no. 100, Princeton University Press, 1981, pp. 151-154. 
G. B. Folland and J. J. Kohn, The Neumann problem for the Cauchy- Riemann complex, 
Annals of Mathematics Studies, no. 75, Princeton University Press, 1972. 
Siqi Fu, Alexander Isaev, Steven Krantz, Finite type conditions on Reinhardt domains, 
Complex Variables: Theory and Application 31 (1996), 357-363. 

Siqi Fu and Emil J. Straube, Compactness of the d-Neumann problem on convex do- 
mains, Jour. Functional Anal. 159(1998), 629-641. 

Bent Fuglede, Finely Harmonic Functions, Lecture Notes in Math. vol. 289, Springer, 
1972. 

, The Dirichlet Laplacian on finely open sets, Potential Anal. 10 (1999), 91-101. 

Theodore W. Gamelin, Uniform algebras and Jensen measures, London Mathematical 
Society Lecture Note Series, no. 32, Cambridge University Press, 1978. 
L. L. Helms, Introduction to potential theory, Wiley-Interscience, 1969. 
G. M. Henkin and A. Iordan, Compactness of the Neumann operator for hyperconvex 
domains with non-smooth B regular boundary, Math. Ann. 307 (1997), 151-168. 
Lars Hormander, L 2 estimates and existence theorems for the d operator, Acta Math- 
cmatica 113 (1965), 89-152. 

J. J. Kohn and L. Nirenberg, Non-coercive boundary value problems, Communications 
on Pure and Applied Mathematics 18 (1965), 443-492. 

Steven G. Krantz, Compactness of the d-Neumann operator, Proceedings of the Amer- 
ican Mathematical Society 103 (1988), no. 4, 1136-1138. 



COMPACTNESS IN THE d-NEUMANN PROBLEM 17 

[Kr92] , Partial differential equations and complex analysis, CRC Press, Boca Raton, 

FL, 1992. 

[La54] Peter D. Lax, Symmetrizable linear transformations, Comm. Pure Appl. Math. 7 
(1954), 633-647. 

[Li85] E. Ligocka, The regularity of the weighted Bergman projections, pp. 197-203 in Seminar 
on deformations, J. Lawrynowicz, editors, Lecture Notes in Math. 1165, Springer, 1985. 

[M97] Peter Matheos, A Hartogs domain with no analytic discs in the boundary for which 
the d-Neumann problem is not compact, Preprint, 1997. (To appear in Jour, of Geom. 
Anal.) 

[Ra84] R. Michael Range, The d-Neumann operator on the unit ball in C™, Math. Ann. 266 
(1984), 449-456. 

[Ro70] R. Tyrrell Rockafellar, Convex analysis, Princeton Mathematical Scries, No. 28 Prince- 
ton University Press, Princeton, N.J. 1970. 

[Sa91] Norberto Salinas, Noncompactness of the d-Neumann problem and Toeplitz C* - 
algebras, Several Complex Variables and Complex Geometry (Eric Bedford, John P. 
D'Angelo, Robert E. Greene, and Steven G. Krantz, eds.), vol. 3, Proceedings of Sym- 
posia in Pure Mathematics, no. 52, American Mathematical Society, 1991, Proceedings 
of the Thirty-seventh Annual Summer Research Institute held at the University of Cal- 
ifornia, Santa Cruz, California, July 10-30, 1989, pp. 329-334. 

[Sa95] , Toeplitz C* -algebras and several complex variables, in Multivariate operator 

theory, Contemp. Math. 185, Amer. Math. Soc, Providence RI, 1995. 

[SSU89] N. Salinas, A. Sheu, and H. Upmeier, Toeplitz operators on pseudoconvex domains and 
foliation C* -algebras, Ann. of Math. 130 (1989), 531-565. 

[Si87] N. Sibony, line classe de domaines pseudoconvexes, Duke Mathematical Journal 55 
(1987), no. 2, 299-319. 

[Si89] , Some aspects of weakly pseudoconvex domains, Several Complex Variables and 

Complex Geometry (Eric Bedford, John P. D'Angelo, Robert E. Greene, and Steven G. 
Krantz, eds.), vol. 1, Proceedings of Symposia in Pure Mathematics, no. 52, Ameri- 
can Mathematical Society, 1991, Proceedings of the Thirty-seventh Annual Summer 
Research Institute held at the University of California, Santa Cruz, California, July 
10-30, 1989, pp. 119-232. 

[St97] Emil J. Straube, Plurisubharmonic functions and subellipticity of the d-Neumann prob- 
lem on non-smooth domains, Math. Res. Lett. 4 (1997), 459-467. 

[V72] U. Vcnugopalkrislma, Fredholm operators associated with strongly pseudoconvex do- 
mains in C n , Jour. Funct. Anal. 9 (1972), 349-373. 

Department of Mathematics, University of Wyoming, Laramie, WY 82071 
E-mail address: sfu0uwyo.edu 



Department of Mathematics, Texas A & M University, College Station, TX 77843 
E-mail address: straubeamath.tamu.edu 



